High-contrast minimum variance imaging method based on deep learning

ABSTRACT

Disclosed is a high-contrast minimum variance imaging method based on deep learning. For the problem of the poor performance of a traditional minimum variance imaging method in terms of ultrasonic image contrast, a deep neural network is applied in order to suppress an off-axis scattering signal in channel data received by an ultrasonic transducer, and after the deep neural network is combined with a minimum variance beamforming method, an ultrasonic image with a higher contrast can be obtained while the resolution performance of the minimum variance imaging method is maintained. In the present method, compared with the traditional minimum variance imaging method, after an apodization weight is calculated, channel data is first processed by using a deep neural network, and weighted stacking of the channel data is then carried out, so that the pixel value of a target imaging point is obtained, thereby forming a complete ultrasonic image.

TECHNICAL FIELD

The present invention belongs to the field of ultrasound imaging, andinnovatively proposes a high-contrast minimum variance imaging methodbased on deep learning, which improves the conventional minimum varianceimaging method.

BACKGROUND

In ultrasound imaging, the existence of off-axis scattering degrades thequality of ultrasound images. In recent years, the combination of deeplearning and ultrasound imaging has become a hot area of research. Withthe strong generalization ability of deep learning, once a deep neuralnetwork has been trained with appropriate training data, the desiredimaging effect can be obtained by using the trained network model.Therefore, with appropriate training data, a deep neural network modelthat can suppress off-axis scatter signals can be obtained.

The minimum variance beamforming method has been widely studied by manyscholars since it was put forward, and many improved techniques havebeen proposed. For example, spatial smoothing can improve the accuracyof estimation of a covariance matrix from data; diagonal loading canimprove the robustness of the minimum variance beamforming method;forward-backward minimum variance beamforming can improve both therobustness of the minimum variance beamforming method and the contrastof the images generated; eigenspace-based beamforming can improve thequality of ultrasound images; and eigenspace-based beamforming combinedwith sign coherence factor can improve both the robustness of theminimum variance beamforming method and the resolution and contrast ofultrasound images. However, the performance of the minimum variancebeamforming method in terms of contrast has never been satisfying, andthe research on the minimum variance beamforming method has also hit abottleneck in improving its performance in terms of contrast. Nowadays,the popular deep learning technology brings new opportunities for theresearch on the minimum variance imaging method. How to combine deeplearning with minimum variance imaging method is worth discussing, as ahigh-contrast minimum variance imaging method based on deep learning haspotential to improve the performance of the minimum variance imagingmethod with regard to contrast.

The high-contrast minimum variance imaging method based on deep learningproposed by the present invention combines the deep neural network withthe minimum variance imaging method, which is a novel minimum varianceimaging method.

SUMMARY

The main purpose of the present invention is to improve the performanceof the minimum variance imaging method in terms of contrast. At present,the research on the combination of deep learning and ultrasound imagingmethods has just started. The present invention adds a deep neuralnetwork operator to the minimum variance imaging method, whichsuppresses off-axis scatter ultrasound signals and improves theperformance of the minimum variance imaging method. The imaging methodof the present invention improves image contrast without compromisingimage resolution, and thus can obtain high-quality ultrasound imageswith good performance in both resolution and contrast.

The purpose of the present invention is achieved by at least one of thefollowing technical schemes.

A high-contrast minimum variance imaging method based on deep learningincludes the following steps:

S1. scanning a target object for ultrasound imaging, generating channeldata composed of echo signals received by reception channels of anultrasound transducer, and performing respective delay operation withregard to different points for imaging to obtain delay channel data;

S2. calculating, according to principles of a minimum variancebeamforming method, an apodization weight vector for channels based onthe delay channel data obtained in S1; and at the same time, performingshort-time Fourier transform on the delay channel data obtained in S1 toobtain frequency domain delay channel data;

S3. suppressing off-axis scatter signals in the frequency domain delaychannel data obtained in S2 by using a deep neural network, to obtainfrequency domain delay channel data with suppressed off-axis scattersignals;

S4. performing inverse short-time Fourier transform on the frequencydomain delay channel data with suppressed off-axis scatter signalsobtained in S3 to obtain delay channel data of each channel that havebeen processed by the deep neural network;

S5. dividing the delay channel data that have been processed by the deepneural network obtained in S4 into corresponding sub-aperture vectors;and

S6. performing weighted summation and calculating an average with theapodization weight vector obtained in S2 and the sub-aperture vectors ofthe delay channel data that have been processed by the deep neuralnetwork obtained in S5 to obtain image pixel values of the correspondingtarget object for ultrasonic imaging, thereby forming a completeultrasound image, which has the advantage of having a high contrast.

Further, in step S1, the scanning a target object for ultrasonicimaging, generating channel data composed of echo signals received byreception channels of an ultrasonic transducer, and then performingdelay operation on the channel data comprises: calculating a delay timeaccording to a position of each target point for imaging, a position ofeach scan line and a position of each reception channel, and mapping thedelay time into a signal subscript, so as to extract a signal valuecorresponding to the target point for imaging in the echo signals of thereception channel and obtain the delay channel data. The delay operationis conventional operation in the ultrasonic imaging process and thuswill not be described in detail here.

Let the number of target points for imaging on one scan line be P andthe number of reception channels be N, so that a P×N delay channel datamatrix is obtained after the delay operation, and let the number of scanlines be L, then a P×N×L delay channel data matrix M₁ is needed forimaging each time, and the subsequent steps will be performed based onthis delay channel data matrix.

Further, in step S2, for one target point for imaging, a delay channeldata vector of length N can be extracted from the delay channel datamatrix M₁; according to the principles of the minimum variancebeamforming method, a spatial smoothing technique is used, that is, afull aperture that contains all the reception channels is divided intoseveral overlapping sub-apertures, a covariance matrix of the delaychannel data in each sub-aperture is calculated individually, and thenan average of the covariance matrices of all the sub-apertures iscalculated. Let the number of channels of each sub-aperture be M, thenthere are N−M+1 sub-apertures in total, and let the sub-aperture vectorsof the delay channel data be x_(i), where i=1, 2, . . . , N−M+1, andx_(i) contains the delay channel data of i-th to (i+M−1)-th receptionchannels. Then, according to the following formula, the covariancematrix of delay channel data in each sub-aperture is calculated and theaverage is calculated to obtain a final estimated covariance matrixR_(cov):

${R_{cov} = {\frac{1}{N - M + 1}{\sum\limits_{i = 1}^{N - M + 1}{x_{i} \cdot x_{i}^{H}}}}};$

where · represents vector multiplication, and H represents conjugatetransposition. The minimum variance beamforming method aims atminimizing the variance of the pixel values of the target points forimaging, and the optimization problem, i.e., to minimize the variance ofthe pixel values of the target points for imaging, is expressed as thefollowing formula:

${\min\limits_{w}{w^{H} \cdot R_{cov} \cdot w}}{{{{s.t.w^{H}} \cdot a} = 1};}$

where a is an all-ones vector, · represents vector multiplication, and wis the apodization weight vector of the channels. The solution to theoptimization problem is:

${w = \frac{R_{cov}^{- 1} \cdot a}{a^{H} \cdot R_{cov}^{- 1} \cdot a}};$

where −1 represents matrix inversion, and represents vectormultiplication; and the weight vector has a length of M, and one weightvector is calculated for each target point for imaging.

After the apodization weight vector of each target point for imaging isobtained, the deep neural network is used to suppress the off-axisscatter signals in the delay channel data, before which the delaychannel data need to be transformed into the frequency domain. Inultrasound imaging, it is advantageous to process signal data infrequency domain.

Further, in step S2, the performing short-time Fourier transform on thedelay channel data obtained in S1 means using discrete short-timeFourier transform to transform the delay channel data from time domainto frequency domain; the discrete short-time Fourier transform meansusing sliding of a window function to divide a long signal into severaloverlapping short signals, which are then subjected to discrete Fouriertransform separately to obtain the frequency domain delay channel data;and the formula of the discrete short-time Fourier transform is asfollows:

${{{{STFT}\left\lbrack y_{p}^{n} \right\rbrack}\left( {m,k} \right)} = {\underset{p = 1}{\sum\limits^{P}}{y_{p}^{n}{w\left( {p - m} \right)}e^{\frac{{- j}2\pi{kp}}{P}}}}};$

where y_(p) ^(n) denotes the delay channel data of an n-th receptionchannel at a p-th target point for imaging on a scan line; p=1, 2, . . ., P; n=1, 2, . . . , N; w(p−m) is the window function, m being a stepsize of the sliding of the window function; k is a serial number of theFourier frequency to be obtained, k having the same value range as p;and j is the imaginary unit.

Further, the window function has a window length of 16, and the delaychannel data of each reception channel on one scan line are a signalvector of length P, so P−16+1 signal vectors of length 16 are obtainedby the sliding of the window function.

According to a symmetry property of the discrete Fourier transform, whena signal sequence is a real-valued sequence, complex amplitudes obtainedfrom the Fourier transform have a conjugate symmetry property, that is,provided that the signal sequence has a length of 16, then 2nd to 8thcomplex amplitudes and 10th to 16th complex amplitudes are conjugatesymmetric, so only the first nine complex amplitudes need to be used.For one scan line, after the short-time Fourier transform of the channeldata, a complex amplitude matrix M₂ with a size of 9×(P−16+1)×N isobtained; and then, real and imaginary parts are separated andrecombined in accordance with the respective reception channels, and adata matrix M₃ with a size of (2×N)×9×(P−16+1) is obtained, which is tobe processed by the deep neural network.

Further, in step S3, the deep neural network used is a feed-forwardfully-connected network with five hidden layers in total, each hiddenlayer having 170 neurons; each frequency corresponds to one network, sothere are nine of the networks in total, and input and output dimensionsof the networks are both 2×N; and the data matrix M₃ obtained for eachscan line is input into the 9 networks for processing according to therespective frequencies, and the obtained frequency domain delay channeldata with suppressed off-axis scatter signals are a data matrix M₄ witha size of (2×N)×9×(P−16+1). A training process of the deep neuralnetwork is as follows: Field II simulation software is used to simulateand generate off-axis scatter signals and non-off-axis scatter signalsto form data of a training set and of a validation set for the deepneural network; the deep neural network is trained with an Adamoptimizer which uses mean and variance of gradients to calculate anupdate step size of network weight parameters, the data of the trainingset are input into the deep neural network in batch while the Adamoptimizer updates the parameters of the deep neural network, a trainingcycle is a period during which all the data of the training set areprocessed once, and the data of the validation set are used to calculatean error on the validation set and change a learning rate accordinglyafter each training cycle ends; and the training process adopts an earlystopping strategy, that is, the training is subjected to early stoppingin response to the error on the validation set having no improvementafter 20 training cycle. The training process takes a long time, butonce the training is done, the deep neural network is able to processdata quickly.

Further, in step S4, the frequency domain delay channel data withsuppressed off-axis scatter signals obtained in S3 is transformed intothe time domain; the data matrix M₄ is recombined into a complexmagnitude matrix M₅ with a size of 9×(P−16+1)×N, which is then expandedto a complex magnitude matrix M₆ with a size of 16×(P−16+1)×N using theconjugate symmetry property; the complex amplitudes of each receptionchannel with a size of 16×(P−16+1) are transformed into time domainsignals using the inverse short-time Fourier transform, therebyobtaining the delay channel data of each channel that have beenprocessed by the deep neural network, with a length of P, and therefore,the delay channel data of L scan lines that have been processed by thedeep neural network are a matrix M₇ with a size of P×N×L, which is thesame as the size of the delay channel data matrix M₁ that has not beenprocessed by the deep neural network.

Further, in step S5, according to the delay channel data that have beenprocessed by the deep neural network obtained in S4, for each targetpoint for imaging on each scan line, a full aperture vector of length Nof the delay channel data that have been processed by the deep neuralnetwork is extracted and divided into N−M+1 corresponding sub-aperturevectors z_(i), where i=1, 2, . . . , N−M+1, and z_(i) contains the delaychannel data that have been processed by the deep neural network of thei-th to the (i+M−1)-th channels.

Further, in step S6, the performing weighted summation and calculatingan average with the apodization weight vector obtained in S2 and thesub-aperture vectors of the delay channel data that have been processedby the deep neural network obtained in S5 is based on the followingformula:

${v = {\frac{1}{N - M + 1}{\sum\limits_{i = 1}^{N - M + 1}{w^{H} \cdot z_{i}}}}};$

where the obtained v is the pixel value of the target point for imaging,and · represents vector multiplication.

Through the above steps, the pixel values of all target points forimaging on all scan lines can be obtained, and then a completeultrasound image can be obtained through subsequent operation ofenvelope detection, logarithmic compression and dynamic range display.The envelope detection and logarithmic compression operation isconventional operation in the ultrasound imaging process and thus willnot be described in detail here.

The high-contrast minimum variance imaging method based on deep learningincludes two main components, i.e., a deep neural network suppressingoff-axis scattering and beamforming. On the basis of the framework ofthe conventional minimum variance imaging method, a deep neural networkoperator is added to suppress the off-axis scatter ultrasound signals,thus improving the quality of ultrasound images.

Compared with the prior art, the present invention mainly has thefollowing advantage: by combining the deep learning technique and theminimum-variance beamforming method for the first time, the deep neuralnetwork, which can suppress off-axis scattering, is integrated into theminimum variance imaging method as an operator, which can improve thecontrast of ultrasound images while maintaining a high resolution.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an imaging process of a high-contrastminimum variance imaging method based on deep learning;

FIG. 2 is a topology diagram of a deep neural network used in theimaging method shown in FIG. 1;

FIG. 3 is an embodiment of a single-point target image output by aconventional minimum variance imaging method;

FIG. 4 is an embodiment of a single-point target image output by ahigh-contrast minimum variance imaging method based on deep learning;

FIG. 5 is a point spread function comparison graph of a single-pointtarget calculated by the high-contrast minimum variance imaging methodbased on deep learning and the conventional minimum variance imagingmethod;

FIG. 6 is an embodiment of a cyst image output by the conventionalminimum variance imaging method;

FIG. 7 is an embodiment of a cyst image output by the high-contrastminimum variance imaging method based on deep learning; and

FIG. 8 is a schematic diagram of regions in and out of a cyst that areselected for calculating image contrast.

DETAILED DESCRIPTION

The specific implementation of the present invention will be furtherdescribed with reference to the drawings and embodiments below, which,however, should not be construed as a limitation on the implementationand scope of protection of the present invention. It should be notedthat details which are not set forth below can be implemented by thoseskilled in the art with reference to the prior art.

EMBODIMENT

As shown in FIG. 1, a high-contrast minimum variance imaging methodbased on deep learning includes the following steps:

S1. scanning a target object for ultrasonic imaging, generating channeldata composed of echo signals received by reception channels of anultrasonic transducer, and performing respective delay operation withregard to different points for imaging to obtain delay channel data;

S2. calculating, according to principles of a minimum variancebeamforming method, an apodization weight vector for channels based onthe delay channel data obtained in S1; and at the same time, performingshort-time Fourier transform on the delay channel data obtained in S1 toobtain frequency domain delay channel data;

S3. suppressing off-axis scatter signals in the frequency domain delaychannel data obtained in S2 by using a deep neural network, to obtainfrequency domain delay channel data with suppressed off-axis scattersignals;

S4. performing inverse short-time Fourier transform on the frequencydomain delay channel data with suppressed off-axis scatter signalsobtained in S3 to obtain delay channel data of each channel that havebeen processed by the deep neural network;

S5. dividing the delay channel data that have been processed by the deepneural network obtained in S4 into corresponding sub-aperture vectors;and

S6. performing weighted summation and calculating an average with theapodization weight vector obtained in S2 and the sub-aperture vectors ofthe delay channel data that have been processed by the deep neuralnetwork obtained in S5 to obtain image pixel values of the correspondingtarget object for ultrasound imaging, thereby forming a completeultrasound image, which has the advantage of having a high contrast.

Further, in step S1, the scanning a target object for ultrasoundimaging, generating channel data composed of echo signals received byreception channels of an ultrasound transducer, and then performingdelay operation on the channel data comprises: calculating a delay timeaccording to a position of each target point for imaging, a position ofeach scan line and a position of each reception channel, and mapping thedelay time into a signal subscript, so as to extract a signal valuecorresponding to the target point for imaging in the echo signals of thereception channel and obtain the delay channel data. The delay operationis conventional operation in the ultrasound imaging process and thuswill not be described in detail here.

Let the number of target points for imaging on one scan line be P andthe number of reception channels be N, so that a P×N delay channel datamatrix is obtained after the delay operation, and let the number of scanlines be L, then a P×N×L delay channel data matrix M₁ is needed forimaging each time, and the subsequent steps will be performed based onthis delay channel data matrix.

Further, in step S2, for one target point for imaging, a delay channeldata vector of length N can be extracted from the delay channel datamatrix M₁; according to the principles of the minimum variancebeamforming method, a spatial smoothing technique is used, that is, afull aperture that contains all the reception channels is divided intoseveral overlapping sub-apertures, a covariance matrix of the delaychannel data in each sub-aperture is calculated individually, and thenan average of the covariance matrices of all the sub-apertures iscalculated. Let the number of channels of each sub-aperture be M, thenthere are N−M+1 sub-apertures in total, and let the sub-aperture vectorsof the delay channel data be x_(i), where i=1, 2, . . . , N−M+1, andx_(i) contains the delay channel data of i-th to (i+M−1)-th receptionchannels. Then, according to the following formula, the covariancematrix of delay channel data in each sub-aperture is calculated and theaverage is calculated to obtain a final estimated covariance matrixR_(cov):

${R_{cov} = {\frac{1}{N - M + 1}{\sum\limits_{i = 1}^{N - M + 1}{x_{i} \cdot x_{i}^{H}}}}};$

where · represents vector multiplication, and H represents conjugatetransposition. The minimum variance beamforming method aims atminimizing the variance of the pixel values of the target points forimaging, and the optimization problem, i.e. to minimize the variance ofthe pixel values of the target points for imaging, is expressed as thefollowing formula:

${\min\limits_{w}{w^{H} \cdot R_{cov} \cdot w}}{{{{s.t.w^{H}} \cdot a} = 1};}$

where a is an all-ones vector, · represents vector multiplication, and wis the apodization weight vector of the channels. The solution to theoptimization problem is:

${w = \frac{R_{cov}^{- 1} \cdot a}{a^{H} \cdot R_{cov}^{- 1} \cdot a}};$

where −1 represents matrix inversion, and · represents vectormultiplication; and the weight vector has a length of M, and one weightvector is calculated for each target point for imaging.

After the apodization weight vector of each target point for imaging isobtained, the deep neural network is used to suppress the off-axisscatter signals in the delay channel data, before which the delaychannel data need to be transformed into the frequency domain. Inultrasound imaging, it is advantageous to process signal data infrequency domain.

Further, in step S2, the performing short-time Fourier transform on thedelay channel data obtained in S1 means using discrete short-timeFourier transform to transform the delay channel data from time domainto frequency domain; the discrete short-time Fourier transform meansusing sliding of a window function to divide a long signal into severaloverlapping short signals, which are then subjected to discrete Fouriertransform separately to obtain the frequency domain delay channel data;and the formula of the discrete short-time Fourier transform is asfollows:

${{{{STFT}\left\lbrack y_{p}^{n} \right\rbrack}\left( {m,k} \right)} = {\sum\limits_{p = 1}^{P}{y_{p}^{n}{w\left( {p - m} \right)}e^{\frac{{- j}2\pi{kp}}{P}}}}};$

where y_(p) ^(n) denotes the delay channel data of an n-th receptionchannel at a p-th target point for imaging on a scan line; p=1, 2, . . ., P; n=1, 2, . . . , N; w(p−m) is the window function, m being a stepsize of the sliding of the window function; k is a serial number of theFourier frequency to be obtained, k having the same value range as p;and j is the imaginary unit. The window function has a window length of16, and the delay channel data of each reception channel on one scanline are a signal vector of length P, so P−16+1 signal vectors of length16 are obtained by the sliding of the window function.

According to a symmetry property of the discrete Fourier transform, whena signal sequence is a real-valued sequence, complex amplitudes obtainedfrom the Fourier transform have a conjugate symmetry property, that is,provided that the signal sequence has a length of 16, then 2nd to 8thcomplex amplitudes and 10th to 16th complex amplitudes are conjugatesymmetric, so only the first nine complex amplitudes need to be used.For one scan line, after the short-time Fourier transform of the channeldata, a complex amplitude matrix M₂ with a size of 9×(P−16+1)×N isobtained; and then, real and imaginary parts are separated andrecombined in accordance with the respective reception channels, and adata matrix M₃ with a size of (2×N)×9×(P−16+1) is obtained, which is tobe processed by the deep neural network.

Further, in step S3, the deep neural network used is a feed-forwardfully-connected network with five hidden layers in total, each hiddenlayer having 170 neurons; each frequency corresponds to one network, sothere are nine of the networks in total, and input and output dimensionsof the networks are both 2×N; and the data matrix M₃ obtained for eachscan line is input into the 9 networks for processing according to therespective frequencies, and the obtained frequency domain delay channeldata with suppressed off-axis scatter signals are a data matrix M₄ witha size of (2×N)×9×(P−16+1). A training process of the deep neuralnetwork is as follows: Field II simulation software is used to simulateand generate off-axis scatter signals and non-off-axis scatter signalsto form data of a training set and of a validation set for the deepneural network, where the training set consists of off-axis scattersignals generated by 5,000 scattering points and non-off-axis scattersignals generated by 5,000 scattering points, the validation setconsists of off-axis scatter signals generated by 1,250 scatteringpoints and non-off-axis scatter signals generated by 1,250 scatteringpoints, and the deep neural network uses the data of the training set tolearn how to identify off-axis scatter signals and suppress them; thedeep neural network is trained with an Adam optimizer which uses meanand variance of gradients to calculate an update step size of networkweight parameters, the data of the training set are input into the deepneural network in batch while the Adam optimizer updates the parametersof the deep neural network, a training cycle is a period during whichall the data of the training set are processed once, and the data of thevalidation set are used to calculate an error on the validation set andchange a learning rate accordingly after each training cycle ends; andthe training process adopts an early stopping strategy, that is, thetraining is subjected to early stopping in response to the error on thevalidation set having no improvement after 20 training cycle. Thetraining process takes a long time, but once the training is done, thedeep neural network is able to process data quickly.

Further, in step S4, the frequency domain delay channel data withsuppressed off-axis scatter signals obtained in S3 is transformed intothe time domain; the data matrix M₄ is recombined into a complexmagnitude matrix M₅ with a size of 9×(P−16+1)×N, which is then expandedto a complex magnitude matrix M₆ with a size of 16×(P−16+1)×N using theconjugate symmetry property; the complex amplitudes of each receptionchannel with a size of 16×(P−16+1) are transformed into time domainsignals using the inverse short-time Fourier transform, therebyobtaining the delay channel data of each channel that have beenprocessed by the deep neural network, with a length of P, and therefore,the delay channel data of L scan lines that have been processed by thedeep neural network are a matrix M₇ with a size of P×N×L, which is thesame as the size of the delay channel data matrix M₁ that has not beenprocessed by the deep neural network.

Further, in step S5, according to the delay channel data that have beenprocessed by the deep neural network obtained in S4, for each targetpoint for imaging on each scan line, a full aperture vector of length Nof the delay channel data that have been processed by the deep neuralnetwork is extracted and divided into N−M+1 corresponding sub-aperturevectors z_(i), where i=1, 2, . . . , N−M+1, and z_(i) contains the delaychannel data that have been processed by the deep neural network of thei-th to the (i+M−1)-th channels.

Further, in step S6, the performing weighted summation and calculatingan average with the apodization weight vector obtained in S2 and thesub-aperture vectors of the delay channel data that have been processedby the deep neural network obtained in S5 is based on the followingformula:

${v = {\frac{1}{N - M + 1}{\sum\limits_{i = 1}^{N - M + 1}{w^{H} \cdot z_{i}}}}};$

where the obtained v is the pixel value of the target point for imaging,and represents vector multiplication.

Through the above steps, the pixel values of all target points forimaging on all scan lines can be obtained, and then a completeultrasound image can be obtained through subsequent operation ofenvelope detection, logarithmic compression and dynamic range display.The envelope detection and logarithmic compression operation isconventional operation in the ultrasound imaging process and thus willnot be described in detail here.

The high-contrast minimum variance imaging method based on deep learningincludes two main components, i.e. a deep neural network suppressingoff-axis scattering and beamforming. On the basis of the steps of theconventional minimum variance imaging method, a deep neural networkoperator is added to suppress the off-axis scatter ultrasound signals,thus improving the contrast of ultrasound images.

The high-contrast minimum variance imaging method based on deep learningproposed by the present invention will eventually be applied toultrasound imaging. In this embodiment, a simulation ultrasound imagingsystem is constructed by three modules, i.e., a data simulation module,a core computing module and an image display module.

1. Data Simulation Module

Field II simulation software is used to simulate a propagation processof ultrasound waves in ultrasound imaging and obtain simulation data. Inthe data simulation module, firstly, physical simulation data aresimulated according to corresponding configuration of a real ultrasoundimaging device, where transmitting and receiving array elements arecreated, and a simulated detection object is created, and then transmitis simulated with scan lines one by one and channel data are received.In this module, a simulated single-point target object and cyst objectare created respectively to observe their imaging effects.

2. Core Computing Module

The core computing module includes an apodization weight calculationmodule and a deep neural network processing module. A training processof the deep neural network is as follows: Field II simulation softwareis used to simulate and generate off-axis scatter signals andnon-off-axis scatter signals to form data of a training set and of avalidation set for the deep neural network, where the training setconsists of off-axis scatter signals generated by 5,000 scatteringpoints and non-off-axis scatter signals generated by 5,000 scatteringpoints, the validation set consists of off-axis scatter signalsgenerated by 1,250 scattering points and non-off-axis scatter signalsgenerated by 1,250 scattering points, and the deep neural network usesthe data of the training set to learn how to identify off-axis scattersignals and suppress them; the deep neural network is trained with anAdam optimizer which uses mean and variance of gradients to calculate anupdate step size of network weight parameters, the data of the trainingset are input into the deep neural network in batch while the Adamoptimizer updates the parameters of the deep neural network, a trainingcycle is a period during which all the data of the training set areprocessed once, and the data of the validation set are used to calculatean error on the validation set and change a learning rate accordinglyafter each training cycle ends; and The training process adopts an earlystopping strategy, that is, the training is subjected to early stoppingin response to the error on the validation set having no improvementafter 20 training cycles. After the received channel data is obtained,the apodization weight vector is calculated by the apodization weightcalculation module according to the principles of the minimum variancebeamforming method. Then, the neural network processing module uses thewell-trained deep neural network to process the channel data to suppressoff-axis scatter signals, and then weighted summation is carried out toobtain the pixel value of each target point for imaging.

3. Image Display Module

In the simulation ultrasound imaging system, after the pixel data areobtained by the core computing module, the image display module usescorresponding decoding programs to perform Hilbert transform,logarithmic compression, grayscale range correction, image depth andwidth calculation and image display operation on the data, and finallyoutputs the image-related data to a corresponding coordinate system todisplay the ultrasound image on a screen.

In the simulated ultrasound imaging, the conventional minimum varianceimaging method and the high-contrast minimum variance imaging methodbased on deep learning are used respectively to calculate the same echodata for comparison.

Scheme and Performance Evaluation:

In the single-point target simulation experiment, there is only onescattering point in the imaging region, which is located at a depth of70 mm. After the data simulation module is invoked to get the channeldata, the core computing module is called to perform calculation for theimage by using the conventional minimum variance imaging method and thehigh-contrast minimum variance imaging method based on deep learning,respectively. The images obtained after the calculation are shown inFIGS. 3 and 4. It can be seen that the high-contrast minimum varianceimaging method based on deep learning can effectively suppress thesignals scattered from scattering points to both sides. Point spreadfunction comparison is shown in FIG. 5. It can be seen that thehigh-contrast minimum variance imaging method based on deep learning notonly maintains the main lobe width of the minimum variance beamformingmethod, namely, without compromising the resolution performance of theminimum variance beamforming method, but also effectively reduces thesidelobe level by about 60 dB.

In the cyst imaging simulation experiment, the imaging depth is 65 mm to75 mm, there are 25 scattering points per cubic millimeter in theimaging region, and the center of a spherical cyst with a diameter of 4mm is located at a depth of 70 mm. After the data simulation module isinvoked to get the channel data, the core computing module is called toperform calculation for the image by using the conventional minimumvariance imaging method and the high-contrast minimum variance imagingmethod based on deep learning, respectively. The images obtained afterthe calculation are shown in FIGS. 6 and 7. The pixels in the cystregion are darker after being processed by the high-contrast minimumvariance imaging method based on deep learning, which suggests that thedeep neural network effectively suppresses the scatter signals in thecyst region. The regions in and out of a cyst that are selected forcalculating image contrast are shown in FIG. 8. Two regions, which havethe same area as the cyst area, are selected on two sides of the cyst,respectively. The average pixel values in the two regions are calculatedand averaged to get the average pixel value out of the cyst region:S_(out)=(S_(out1)+S_(out2))/2, where S_(out1) is the average pixel valuein region 1 out of the cyst region, and S_(out2) is the average pixelvalue in region 2 out of the cyst region. S_(in) is the average pixelvalue in the cyst region, and then the contrast CR is calculated asfollows:

${CR} = {\frac{S_{out} - S_{in}}{S_{out}}.}$

The calculated image contrast is 0.3568 in FIG. 6, and 0.3900 in FIG. 7,which shows that the high-contrast minimum variance imaging method basedon deep learning improves the contrast. According to the above data,compared with the conventional minimum variance imaging method, thehigh-contrast minimum variance imaging method based on deep learning ofthe present invention improves the image contrast without compromisingthe image resolution.

The embodiment describes the improvement of ultrasound image quality bydesigning and evaluating the high-contrast minimum variance imagingmethod based on deep learning in simulated ultrasound imaging. Theevaluation results show that the contrast of the ultrasound image isimproved by using the high-contrast minimum variance imaging methodbased on deep learning compared with the conventional minimum varianceimaging method.

1. A high-contrast minimum variance imaging method based on deeplearning, comprising the following steps: S1. scanning a target objectfor ultrasound imaging, generating a channel data composed of echosignals received by reception channels of an ultrasound transducer, andperforming a respective delay operation with regard to different pointsfor imaging to obtain a delay channel data; S2. calculating, accordingto principles of a minimum variance beamforming method, an apodizationweight vector for channels based on the delay channel data obtained inthe S1; and at the same time, performing short-time Fourier transform onthe delay channel data obtained in the S1 to obtain a frequency domaindelay channel data; S3. suppressing off-axis scatter signals in thefrequency domain delay channel data obtained in the S2 by using a deepneural network, to obtain the frequency domain delay channel data withthe off-axis scatter signals, which are suppressed; S4. performing aninverse short-time Fourier transform on the frequency domain delaychannel data with the off-axis scatter signals, which are suppressed,obtained in the S3 to obtain the delay channel data of each channel thathave been processed by the deep neural network; S5. dividing the delaychannel data that have been processed by the deep neural networkobtained in the S4 into corresponding sub-aperture vectors; and S6.performing a weighted summation and calculating an average with theapodization weight vector obtained in the S2 and sub-aperture vectors ofthe delay channel data that have been processed by the deep neuralnetwork obtained in the S5 to obtain image pixel values of the targetobject for ultrasound imaging, which is corresponded, thereby forming acomplete ultrasound image.
 2. The high-contrast minimum variance imagingmethod based on deep learning of claim 1, wherein in the step S1, thescanning the target object for ultrasound imaging, generating thechannel data composed of the echo signals received by the receptionchannels of the ultrasound transducer, and then performing a delayoperation on the channel data comprises: calculating a delay timeaccording to a position of each of target points for imaging, a positionof each of scan lines and a position of each reception channel, andmapping the delay time into a signal subscript, so as to extract asignal value corresponding to the target points for imaging in the echosignals of a reception channel and obtain the delay channel data; andlet the number of the target points for imaging on one of the scan linesbe P and the number of the reception channels be N, so that a P×N delaychannel data matrix is obtained after the delay operation, and let thenumber of the scan lines be L, then a P×N×L delay channel data matrix M1is needed for imaging each time, and subsequent steps will be performedbased on this delay channel data matrix.
 3. The high-contrast minimumvariance imaging method based on deep learning of claim 1, wherein inthe step S2, for one of target points for imaging, a delay channel datavector of a length N can be extracted from a delay channel data matrixM1; according to the principles of the minimum variance beamformingmethod, a spatial smoothing technique is used, that is, a full aperturethat contains all the reception channels is divided into severaloverlapping sub-apertures, a covariance matrix of the delay channel datain each of sub-apertures is calculated individually, and then an averageof the covariance matrices of all the sub-apertures is calculated; letthe number of the channels of each of the sub-apertures be M, then thereare N−M+1 sub-apertures in total, and let the sub-aperture vectors ofthe delay channel data be xi, where i=1, 2, . . . , N−M+1, and xicontains the delay channel data of i-th to (i+M−1)-th receptionchannels; then, according to the following formula, the covariancematrix of the delay channel data in each of the sub-apertures iscalculated and an average is calculated to obtain a final estimatedcovariance matrix Rcov:${R_{cov} = {\frac{1}{N - M + 1}{\sum\limits_{i = 1}^{N - M + 1}{x_{i} \cdot x_{i}^{H}}}}};$where · represents a vector multiplication, and H represents a conjugatetransposition; a minimum variance beamforming method aims at minimizinga variance of pixel values of the target points for imaging, and aoptimization problem, i.e., to minimize the variance of the pixel valuesof the target points for imaging, is expressed as the following formula:${\min\limits_{w}{w^{H} \cdot R_{cov} \cdot w}}{{{{s.t.w^{H}} \cdot a} = 1};}$where a is an all-ones vector, · represents the vector multiplication,and w is the apodization weight vector of the channels; and the solutionto the optimization problem is:${w = \frac{R_{cov}^{- 1} \cdot a}{a^{H} \cdot R_{cov}^{- 1} \cdot a}};$where −1 represents a matrix inversion, and represents the vectormultiplication; and a weight vector has a length of M, and one of theweight vector is calculated for each of the target points for imaging.4. The high-contrast minimum variance imaging method based on deeplearning of claim 1, wherein in the step S2, the performing a short-timeFourier transform on the delay channel data obtained in the S1 meansusing a discrete short-time Fourier transform to transform the delaychannel data from a time domain to a frequency domain; the discreteshort-time Fourier transform means using a sliding of a window functionto divide a long signal into several overlapping short signals, whichare then subjected to a discrete Fourier transform separately to obtainthe frequency domain delay channel data; and the formula of the discreteshort-time Fourier transform is as follows:${{{{STFT}\left\lbrack y_{p}^{n} \right\rbrack}\left( {m,k} \right)} = {\sum\limits_{p = 1}^{P}{y_{p}^{n}{w\left( {p - m} \right)}e^{\frac{{- j}2\pi{kp}}{P}}}}};$where y_(p) ^(n) denotes the delay channel data of an n-th receptionchannel at a p-th target point for imaging on a scan line; p=1, 2, . . ., P; n=1, 2, . . . , N; w(p−m) is the window function, m being a stepsize of the sliding of the window function; k is a serial number of aFourier frequency to be obtained, k having the same value range as p;and j is an imaginary unit.
 5. The high-contrast minimum varianceimaging method based on deep learning of claim 4, wherein the windowfunction has a window length of 16, and the delay channel data of eachreception channel on one scan line are a signal vector of a length P, soP−16+1 signal vectors of length 16 are obtained by the sliding of thewindow function; and according to a symmetry property of the discreteFourier transform, when a signal sequence is a real-valued sequence,complex amplitudes obtained from a Fourier transform have a conjugatesymmetry property, that is, provided that the signal sequence has alength of 16, then 2nd to 8th of the complex amplitudes and 10th to 16thof the complex amplitudes are conjugate symmetric, so only first nine ofthe complex amplitudes need to be used; for the one scan line, after theshort-time Fourier transform of the channel data, a complex amplitudematrix M2 with a size of 9×(P−16+1)×N is obtained; and then, real andimaginary parts are separated and recombined in accordance with thereception channels, which are respective, and a data matrix M3 with asize of (2×N)×9×(P−16+1) is obtained, which is to be processed by thedeep neural network.
 6. The high-contrast minimum variance imagingmethod based on deep learning of claim 1, wherein in the step S3, thedeep neural network used is a feed-forward fully-connected network withfive hidden layers in total, each of the hidden layers having 170neurons; each frequency corresponds to one of networks, so there arenine of the networks in total, and input and output dimensions of thenetworks are both 2×N; a data matrix M3 obtained for each scan line isinput into the nine networks for processing according to respectivefrequencies, and the frequency domain delay channel data, which isobtained, with the off-axis scatter signals, which are suppressed, are adata matrix M4 with a size of (2×N)×9×(P−16+1); and a training processof the deep neural network is as follows: Field II simulation softwareis used to simulate and generate the off-axis scatter signals andnon-off-axis scatter signals to form a data of a training set and of avalidation set for the deep neural network; the deep neural network istrained with an Adam optimizer which uses a mean and a variance ofgradients to calculate an update step size of network weight parameters,the data of the training set are input into the deep neural network inbatch while the Adam optimizer updates parameters of the deep neuralnetwork, a training cycle is a period during which all the data of thetraining set are processed once, and the data of the validation set areused to calculate an error on the validation set and change a learningrate accordingly after each training cycle ends; and the trainingprocess adopts an early stopping strategy, that is, a training issubjected to early stopping in response to the error on the validationset having no improvement after 20 training cycles.
 7. The high-contrastminimum variance imaging method based on deep learning of claim 1,wherein in the step S4, the frequency domain delay channel data with theoff-axis scatter signals, which are suppressed, obtained in the step S3is transformed into a time domain; a data matrix M₄ is recombined into acomplex magnitude matrix M5 with a size of 9×(P−16+1)×N, which is thenexpanded to a complex magnitude matrix M6 with a size of 16×(P−16+1)×Nusing a conjugate symmetry property; the complex amplitudes of eachreception channel with a size of 16×(P−16+1) are transformed into timedomain signals using the inverse short-time Fourier transform, therebyobtaining the delay channel data of each channel that have beenprocessed by the deep neural network, with a length of P, and therefore,the delay channel data of L scan lines that have been processed by thedeep neural network are a matrix M7 with a size of P×N×L, which is thesame as a size of a delay channel data matrix M₁ that has not beenprocessed by the deep neural network.
 8. The high-contrast minimumvariance imaging method based on deep learning of claim 1, wherein inthe step S5, according to the delay channel data that have beenprocessed by the deep neural network obtained in the S4, for each targetpoint for imaging on each scan line, a full aperture vector of length Nof the delay channel data that have been processed by the deep neuralnetwork is extracted and divided into N−M+1 of the correspondingsub-aperture vectors zi, where i=1, 2, . . . , N−M+1, and zi containsthe delay channel data that have been processed by the deep neuralnetwork of i-th to (i+M−1)-th of the channels.
 9. The high-contrastminimum variance imaging method based on deep learning of claim 1,wherein in step S6, the performing the weighted summation andcalculating the average with the apodization weight vector obtained inthe S2 and the sub-aperture vectors of the delay channel data that havebeen processed by the deep neural network obtained in the S5 is based onthe following formula:${v = {\frac{1}{N - M + 1}{\sum\limits_{i = 1}^{N - M + 1}{w^{H} \cdot z_{i}}}}};$where the obtained v is a pixel value of target points for imaging, and· represents a vector multiplication; and pixel values of all the targetpoints for imaging then constitute the complete ultrasound image, whichhas the advantage of having a high contrast.